On the Limits of GPU Acceleration

This paper throws a small “wet blanket” on the hot topic of GPGPU acceleration, based on experience analyzing and tuning both multithreaded CPU and GPU implementations of three computations in scientific computing. These computations—(a) iterative sparse linear solvers; (b) sparse Cholesky factorization; and (c) the fast multipole method—exhibit complex behavior and vary in computational intensity and memory reference irregularity. In each case, algorithmic analysis and prior work might lead us to conclude that an idealized GPU can deliver better performance, but we find that for at least equal-effort CPU tuning and consideration of realistic workloads and calling-contexts, we can with two modern quad-core CPU sockets roughly match one or two GPUs in performance. Our conclusions are not intended to dampen interest in GPU acceleration; on the contrary, they should do the opposite: they partially illuminate the boundary between CPU and GPU performance, and ask architects to consider application contexts in the design of future coupled on-die CPU/GPU processors. 1 Our Position and Its Limitations We have over the past year been interested in the analysis, implementation, and tuning of a variety of irregular computations arising in computational science and engineering applications, for both multicore CPUs and GPGPU platforms [4, 11, 5, 16, 1]. In reflecting on this experience, the following question arose: What is the boundary between computations that can and cannot be effectively accelerated by GPUs, relative to general-purpose multicore CPUs within a roughly comparable power footprint? Though we do not claim a definitive answer to this question, we believe our preliminary findings might surprise the broader community of application development teams whose charge it is to decide whether and how much effort to expend on GPGPU code development. Position. Our central aim is to provoke a more realistic discussion about the ultimate role of GPGPU accelerators in applications. In particular, we argue that, for a moderately complex class of “irregular” computations, even well-tuned GPGPU accelerated implementations on currently available systems will deliver performance that is, roughly speaking, only comparable to well-tuned code for general-purpose multicore CPU systems, within a roughly comparable power footprint. Put another way, adding a GPU is equivalent in performance to simply adding one or perhaps two more multicore CPU sockets. Thus, one might reasonably ask whether this level of performance increase is worth the potential productivity loss from adoption of a new programming model and re-tuning for the accelerator. Our discussion considers (a) iterative solvers for sparse linear systems; (b) direct solvers for sparse linear systems; and (c) the fast multipole method for particle systems. These appear in traditional high-performance scientific computing applications, but are also of increasing importance in graphics, physics-based games, and large-scale machine learning problems. Threats to validity. Our conclusions represent our interpretation of the data. By way of full-disclosure upfront, we acknowledge at least the following three major weaknesses in our position. • (Threat 1) Our perspective comes from relatively narrow classes of applications. These computations come from traditional HPC applications. • (Threat 2) Some conclusions are drawn from partial results. Our work is very much on-going, and we are carefully studying our GPU codes to ensure that we have not missed additional tuning opportunities.